Advancing Real-time Pandemic Forecasting Using Large Language Models: A COVID-19 Case Study (preprint)

Forecasting the short-term spread of an ongoing disease outbreak is a formidable challenge due to the complexity of contributing factors, some of which can be characterized through interlinked, multi-modality variables such as epidemiological time series data, viral biology, population demographics, and the intersection of public policy and human behavior. Existing forecasting model frameworks struggle with the multifaceted nature of relevant data and robust results translation, which hinders their performances and the provision of actionable insights for public health decision-makers. Our work introduces PandemicLLM, a novel framework with multi-modal Large Language Models (LLMs) that reformulates real-time forecasting of disease spread as a text reasoning problem, with the ability to incorporate real-time, complex, non-numerical information that previously unattainable in traditional forecasting models. This approach, through a unique AI-human cooperative prompt design and time series representation learning, encodes multi-modal data for LLMs. The model is applied to the COVID-19 pandemic, and trained to utilize textual public health policies, genomic surveillance, spatial, and epidemiological time series data, and is subsequently tested across all 50 states of the U.S. Empirically, PandemicLLM is shown to be a high-performing pandemic forecasting framework that effectively captures the impact of emerging variants and can provide timely and accurate predictions. The proposed PandemicLLM opens avenues for incorporating various pandemic-related data in heterogeneous formats and exhibits performance benefits over existing models. This study illuminates the potential of adapting LLMs and representation learning to enhance pandemic forecasting, illustrating how AI innovations can strengthen pandemic responses and crisis management in the future.

Association between vaccination rates and severe COVID-19 health outcomes in the United States: a population-level statistical analysis (preprint)

Vaccine development and distribution have been at the forefront of efforts to combat the COVID-19 pandemic. As the vaccines have been widely adopted by the population, uncertainties around their effectiveness resulting from the emergence of new variants and other confounding factors make it challenging to determine their real-world impact, which is critical for understanding risk, informing public health policies, and mitigating the impact of COVID-19. We analyzed the association between time-dependent vaccination rates and COVID-19 severity for 48 states in the U.S. using Generalized Additive Models (GAMs). We controlled for additional dynamic factors such as testing rates, purpose-specific travel behaviors, underlying population immunity, and policy, and critical static factors such as comorbidities, social vulnerability, race, and state healthcare expenditures. We used SARS-CoV-2 genomic surveillance data to model the different COVID-19 variant driven waves separately, and evaluate if there is a changing role of the potential drivers of severity over time and across waves. Our study revealed a strong and statistically significant negative association between vaccine uptake and COVID-19 severity across each variant wave. Results also showed that booster shots offered additional protection against severe diseases during the Omicron wave. Additionally, higher underlying population immunity based on previous infection rates are shown to be associated with reduced COVID-19 severity. Full-service restaurant visits are associated with increased COVID-19 severity for the pre-Delta and Delta waves, while office of physician visits are associated with increased COVID-19 severity for the Omicron wave. Moreover, the states with higher government policy index scores have lower COVID-19 severity. Regarding static variables, the social vulnerability index, and the proportion of adults at high risk exhibit positive associations with COVID-19 severity, while Medicaid spending per person exhibits a negative association with COVID-19 severity. Despite the emergence of new variants, vaccines remain highly effective at reducing severe outcomes of COVID-19. Therefore, given the ongoing threat posed by COVID-19, vaccines remain a critical line of defense for protecting the public and preventing burden on healthcare systems.

A Deep Learning Approach to Forecast Short-Term COVID-19 Cases and Deaths in the US (preprint)

Since the US reported its first COVID-19 case on January 21, 2020, the science community has been applying various techniques to forecast incident cases and deaths. To date, providing an accurate and robust forecast at a high spatial resolution has proved challenging, even in the short term. Here we present a novel multi-stage deep learning model to forecast the number of COVID-19 cases and deaths for each US state at a weekly level for a forecast horizon of 1 to 4 weeks. The model is heavily data driven, and relies on epidemiological, mobility, survey, climate, and demographic. We further present results from a case study that incorporates SARS-CoV-2 genomic data (i.e. variant cases) to demonstrate the value of incorporating variant cases data into model forecast tools. We implement a rigorous and robust evaluation of our model – specifically we report on weekly performance over a one-year period based on multiple error metrics, and explicitly assess how our model performance varies over space, chronological time, and different outbreak phases. The proposed model is shown to consistently outperform the CDC ensemble model for all evaluation metrics in multiple spatiotemporal settings, especially for the longer-term (3 and 4 weeks ahead) forecast horizon. Our case study also highlights the potential value of virus genomic data for use in short-term forecasting to identify forthcoming surges driven by new variants. Based on our findings, the proposed forecasting framework improves upon the available forecasting tools currently used to support public health decision making with respect to COVID-19 risk. Research in context Evidence before this study A systematic review of the COVID-19 forecasting and the EPIFORGE 2020 guidelines reveal the lack of consistency, reproducibility, comparability, and quality in the current COVID-19 forecasting literature. To provide an updated survey of the literature, we carried out our literature search on Google Scholar, PubMed, and medRxi , using the terms “Covid-19,” “SARS-CoV-2,” “coronavirus,” “short-term,” “forecasting,” and “genomic surveillance.” Although the literature includes a significant number of papers, it remains lacking with respect to rigorous model evaluation, interpretability and translation. Furthermore, while SARS-CoV-2 genomic surveillance is emerging as a vital necessity to fight COVID-19 (i.e. wastewater sampling and airport screening), to our knowledge, no published forecasting model has illustrated the value of virus genomic data for informing future outbreaks. Added value of this study We propose a multi-stage deep learning model to forecast COVID-19 cases and deaths with a horizon window of four weeks. The data driven model relies on a comprehensive set of input features, including epidemiological, mobility, behavioral survey, climate, and demographic. We present a robust evaluation framework to systematically assess the model performance over a one-year time span, and using multiple error metrics. This rigorous evaluation framework reveals how the predictive accuracy varies over chronological time, space, and outbreak phase. Further, a comparative analysis against the CDC ensemble, the best performing model in the COVID-19 ForecastHub, shows the model to consistently outperform the CDC ensemble for all evaluation metrics in multiple spatiotemporal settings, especially for the longer forecasting windows. We also conduct a feature analysis, and show that the role of explanatory features changes over time. Specifically, we note a changing role of climate variables on model performance in the latter half of the study period. Lastly, we present a case study that reveals how incorporating SARS-CoV-2 genomic surveillance data may improve forecasting accuracy compared to a model without variant cases data. Implications of all the available evidence Results: from the robust evaluation analysis highlight extreme model performance variability over time and space, and suggest that forecasting models should be accompanied with specifications on the conditions under which they perform best (and worst), in order to maximize their value and utility in aiding public health decision making. The feature analysis reveals the complex and changing role of factors contributing to COVID-19 transmission over time, and suggests a possible seasonality effect of climate on COVID-19 spread, but only after August 2021. Finally, the case study highlights the added value of using genomic surveillance data in short-term epidemiological forecasting models, especially during the early stage of new variant introductions.

Unified real-time environmental-epidemiological data for multiscale modeling of the COVID-19 pandemic (preprint)

An impressive number of COVID-19 data catalogs exist. None, however, are optimized for data science applications, e.g ., inconsistent naming and data conventions, uneven quality control, and lack of alignment between disease data and potential predictors pose barriers to robust modeling and analysis. To address this gap, we generated a unified dataset that integrates and implements quality checks of the data from numerous leading sources of COVID-19 epidemiological and environmental data. We use a globally consistent hierarchy of administrative units to facilitate analysis within and across countries. The dataset applies this unified hierarchy to align COVID-19 case data with a number of other data types relevant to understanding and predicting COVID-19 risk, including hydrometeorological data, air quality, information on COVID-19 control policies, and key demographic characteristics.

Social Distancing is Effective at Mitigating COVID-19 Transmission in the United States (preprint)

COVID-19 is present in every state and over 90 percent of all counties in the United States. Decentralized government efforts to reduce spread, combined with the complex dynamics of human mobility and the variable intensity of local outbreaks makes assessing the effect of large-scale social distancing on COVID-19 transmission in the U.S.a challenge. We generate a novel metric to represent social distancing behavior derived from mobile phone data and examine its relationship with COVID-19 case reports at the county level. Our analysis reveals that social distancing is strongly correlated with decreased COVID-19 case growth rates for the 25 most affected counties in the United States, with a lag period consistent with the incubation time of SARS-CoV-2. We also demonstrate evidence that social distancing was already under way in many U.S. counties before state or local-level policies were implemented. This study strongly supports social distancing as an effective way to mitigate COVID-19 transmission in the United States.